{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 如何使用千帆 Python SDK 搭配预置大模型服务进行批量推理\n",
    "\n",
    "在 0.2.8 版本中，千帆 Python SDK 增加了对批量推理的支持，使用该功能需要视情况开通千帆的模型服务，确保您的账号可以调用您想进行批量推理的服务。\n",
    "\n",
    "# 准备工作\n",
    "\n",
    "在开始之前，请确保你的千帆 Python SDK 已经升级到了 0.2.8 及以上版本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "pip install -U \"qianfan>=0.2.8\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "并且在环境变量中设置好 Access Key 与 Secret Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import logging\n",
    "import os\n",
    "\n",
    "from qianfan.utils import enable_log\n",
    "\n",
    "os.environ['QIANFAN_ACCESS_KEY'] = 'your_access_key'\n",
    "os.environ['QIANFAN_SECRET_KEY'] = 'your_secret_key'\n",
    "\n",
    "os.environ[\"QIANFAN_QPS_LIMIT\"] = \"1\"\n",
    "os.environ['QIANFAN_LLM_API_RETRY_COUNT'] = \"3\"\n",
    "\n",
    "# 选择打印出来的日志等级，目前打印出 info 级别\n",
    "enable_log(logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正文\n",
    "\n",
    "为了开始批量推理，我们首先需要获取到用于做批量推理输入的数据集文件，并且指定用做推理输入的列名"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [01-05 16:17:53] dataset.py:466 [t:8120441664]: no data source was provided, construct\n",
      "[INFO] [01-05 16:17:53] dataset.py:344 [t:8120441664]: construct a file data source from path: data_file/qa_pair.csv, with args: {'input_columns': ['prompt'], 'reference_column': 'response'}\n",
      "[INFO] [01-05 16:17:53] data_source.py:280 [t:8120441664]: use format type FormatType.Csv\n",
      "[INFO] [01-05 16:17:53] dataset.py:905 [t:8120441664]: list local dataset data by 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'prompt': '地球的自转周期是多久？', 'response': '大约24小时'}\n"
     ]
    }
   ],
   "source": [
    "from qianfan.dataset import Dataset\n",
    "\n",
    "dataset_file_path = \"data_file/qa_pair.csv\"\n",
    "dataset_input_column_list = [\"prompt\"]\n",
    "\n",
    "# 预期输出列列名，当数据集为对话类数据集时必填，为非对话数据集时选填。\n",
    "# 对应列的数据会在推理结果中出现\n",
    "reference_column = \"response\"\n",
    "\n",
    "ds = Dataset.load(data_file=dataset_file_path, input_columns=dataset_input_column_list, reference_column=reference_column)\n",
    "\n",
    "# 预览数据格式\n",
    "print(ds.list(0))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在导入之后，用户可以根据自己的需求，传入不同的参数来使用不同的方式进行推理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6289698816]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6357004288]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6323351552]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6306525184]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6340177920]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6390657024]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6407483392]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6441136128]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6373830656]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:6424309760]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10754224128]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10771050496]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10787876864]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10804703232]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10855182336]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10838355968]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10821529600]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10872008704]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10888835072]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10905661440]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10939314176]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10922487808]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10956140544]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10972966912]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11006619648]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:10989793280]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11023446016]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11057098752]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11040272384]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11090751488]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11073925120]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11124404224]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11158056960]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11174883328]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11191709696]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11107577856]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11141230592]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11208536064]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11242188800]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11225362432]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11259015168]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11275841536]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11309494272]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11326320640]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11343147008]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11376799744]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11359973376]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11292667904]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11410452480]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11393626112]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11427278848]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11444105216]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11460931584]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11494584320]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11477757952]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11528237056]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11545063424]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11511410688]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11561889792]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11578716160]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11595542528]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11629195264]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11612368896]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11679674368]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11646021632]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11696500736]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11662848000]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11763806208]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11713327104]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11746979840]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11814285312]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11730153472]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11831111680]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11797458944]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11780632576]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11847938048]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11881590784]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11864764416]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11948896256]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11915243520]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11932069888]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11898417152]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11982548992]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11999375360]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:11965722624]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:50] openapi_requestor.py:135 [t:12016201728]: requesting llm api endpoint: /chat/completions_pro\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6407483392]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:208 [t:6407483392]: trying to refresh access_token for ak `BWMBe7***`\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6424309760]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6289698816]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6373830656]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6390657024]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10787876864]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6306525184]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6357004288]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6441136128]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6340177920]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:6323351552]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10804703232]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10754224128]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10771050496]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:221 [t:6407483392]: sucessfully refresh access_token\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10821529600]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10855182336]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10838355968]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10872008704]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10922487808]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10956140544]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10888835072]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10939314176]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10905661440]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11242188800]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11191709696]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11090751488]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11309494272]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11343147008]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11073925120]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11410452480]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11006619648]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10989793280]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11158056960]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11040272384]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11174883328]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11225362432]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11208536064]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11141230592]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11595542528]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11393626112]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11730153472]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11881590784]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11023446016]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11376799744]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11511410688]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11477757952]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11444105216]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11460931584]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11494584320]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:10972966912]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11359973376]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11696500736]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11864764416]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11629195264]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11545063424]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11646021632]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11292667904]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11965722624]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11915243520]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11124404224]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11612368896]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11427278848]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11814285312]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11932069888]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11746979840]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11259015168]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11662848000]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11898417152]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11275841536]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11982548992]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11713327104]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11057098752]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11780632576]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11107577856]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11763806208]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11948896256]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11326320640]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11679674368]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11797458944]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11561889792]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11578716160]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11831111680]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:12016201728]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11528237056]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11847938048]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:02:51] oauth.py:301 [t:11999375360]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-28 13:04:22] base.py:84 [t:11679674368]: All tasks finished, exeutor will be shutdown\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 0 的第 1 片回包请求响应时延: 3.234425, 传输完成时延: 4.2443050830625\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 1 的第 1 片回包请求响应时延: 4.574222, 传输完成时延: 9.60157487494871\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 2 的第 1 片回包请求响应时延: 8.543243, 传输完成时延: 17.589555207872763\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 3 的第 1 片回包请求响应时延: 6.041158, 传输完成时延: 14.08376762480475\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 4 的第 1 片回包请求响应时延: 3.740261, 传输完成时延: 9.772562291007489\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 5 的第 1 片回包请求响应时延: 7.740179, 传输完成时延: 9.751538500189781\n",
      "[INFO] [12-28 13:04:22] dataset.py:1727 [t:8120441664]: 数据 6 的第 1 片回包请求响应时延: 0.41984, 传输完成时延: 3.4382537910714746\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 7 的第 1 片回包请求响应时延: 3.60314, 传输完成时延: 16.66318070795387\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 8 的第 1 片回包请求响应时延: 9.908252, 传输完成时延: 9.91030141687952\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 9 的第 1 片回包请求响应时延: 7.378038, 传输完成时延: 14.415524458047003\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 10 的第 1 片回包请求响应时延: 5.511804, 传输完成时延: 16.5632968749851\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 11 的第 1 片回包请求响应时延: 6.965926, 传输完成时延: 19.021090291906148\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 12 的第 1 片回包请求响应时延: 6.499118, 传输完成时延: 10.52211062493734\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 13 的第 1 片回包请求响应时延: 3.688757, 传输完成时延: 13.739469917025417\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 14 的第 1 片回包请求响应时延: 2.908143, 传输完成时延: 16.91922508389689\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 15 的第 1 片回包请求响应时延: 10.171682, 传输完成时延: 26.17449662508443\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 16 的第 1 片回包请求响应时延: 5.693887, 传输完成时延: 20.69838562514633\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 17 的第 1 片回包请求响应时延: 4.607113, 传输完成时延: 21.604456124827266\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 18 的第 1 片回包请求响应时延: 7.676248, 传输完成时延: 27.651436042040586\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 19 的第 1 片回包请求响应时延: 5.423862, 传输完成时延: 27.386275499826297\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 20 的第 1 片回包请求响应时延: 2.987445, 传输完成时延: 20.976216666866094\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 21 的第 1 片回包请求响应时延: 7.067154, 传输完成时延: 28.037593957968056\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 22 的第 1 片回包请求响应时延: 7.746403, 传输完成时延: 26.72343062516302\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 23 的第 1 片回包请求响应时延: 3.227123, 传输完成时延: 52.13198395888321\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 24 的第 1 片回包请求响应时延: 7.918166, 传输完成时延: 38.82817762508057\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 25 的第 1 片回包请求响应时延: 5.485603, 传输完成时延: 35.40898966696113\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 26 的第 1 片回包请求响应时延: 4.718257, 传输完成时延: 46.60138166719116\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 27 的第 1 片回包请求响应时延: 6.887874, 传输完成时延: 39.7476761250291\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 28 的第 1 片回包请求响应时延: 8.379974, 传输完成时延: 79.34304862492718\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 29 的第 1 片回包请求响应时延: 5.105301, 传输完成时延: 33.030827125068754\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 30 的第 1 片回包请求响应时延: 5.765254, 传输完成时延: 30.68042966700159\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 31 的第 1 片回包请求响应时延: 5.834488, 传输完成时延: 78.80217420798726\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 32 的第 1 片回包请求响应时延: 4.607191, 传输完成时延: 63.535263708094135\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 33 的第 1 片回包请求响应时延: 11.929546, 传输完成时延: 48.803849374875426\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 34 的第 1 片回包请求响应时延: 11.048886, 传输完成时延: 42.90446162503213\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 35 的第 1 片回包请求响应时延: 4.576561, 传输完成时延: 38.4403063331265\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 36 的第 1 片回包请求响应时延: 6.773861, 传输完成时延: 30.688418000005186\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 37 的第 1 片回包请求响应时延: 16.707138, 传输完成时延: 52.576665333937854\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 38 的第 1 片回包请求响应时延: 15.647047, 传输完成时延: 50.51567520783283\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 39 的第 1 片回包请求响应时延: 5.006202, 传输完成时延: 27.915414666058496\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 40 的第 1 片回包请求响应时延: 9.196276, 传输完成时延: 74.13927054195665\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 41 的第 1 片回包请求响应时延: 6.597782, 传输完成时延: 74.54709783289582\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 42 的第 1 片回包请求响应时延: 16.792429, 传输完成时延: 72.71220629196614\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 43 的第 1 片回包请求响应时延: 5.471915, 传输完成时延: 31.39039762504399\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 44 的第 1 片回包请求响应时延: 3.819644, 传输完成时延: 79.79934741579928\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 45 的第 1 片回包请求响应时延: 7.261088, 传输完成时延: 34.181920083006844\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 46 的第 1 片回包请求响应时延: 6.009063, 传输完成时延: 55.91799929179251\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 47 的第 1 片回包请求响应时延: 13.041002, 传输完成时延: 55.92577166692354\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 48 的第 1 片回包请求响应时延: 8.798931, 传输完成时延: 47.67474554106593\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 49 的第 1 片回包请求响应时延: 4.703139, 传输完成时延: 33.628974499879405\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 50 的第 1 片回包请求响应时延: 7.220547, 传输完成时延: 68.14947033300996\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 51 的第 1 片回包请求响应时延: 14.232726, 传输完成时延: 60.13044429104775\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 52 的第 1 片回包请求响应时延: 7.136343, 传输完成时延: 54.03630566690117\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 53 的第 1 片回包请求响应时延: 7.542619, 传输完成时延: 52.43595845904201\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 54 的第 1 片回包请求响应时延: 7.400525, 传输完成时延: 55.30382708297111\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 55 的第 1 片回包请求响应时延: 15.098678, 传输完成时延: 58.98808499984443\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 56 的第 1 片回包请求响应时延: 6.41947, 传输完成时延: 89.4290900840424\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 57 的第 1 片回包请求响应时延: 16.844117, 传输完成时延: 70.76616220804863\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 58 的第 1 片回包请求响应时延: 5.20866, 传输完成时延: 84.19920783303678\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 59 的第 1 片回包请求响应时延: 4.851056, 传输完成时延: 84.84522137511522\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 60 的第 1 片回包请求响应时延: 16.407721, 传输完成时延: 54.28196416608989\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 61 的第 1 片回包请求响应时延: 19.433108, 传输完成时延: 79.36152283311822\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 62 的第 1 片回包请求响应时延: 17.850875, 传输完成时延: 70.76665658294223\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 63 的第 1 片回包请求响应时延: 8.700166, 传输完成时延: 63.61991699994542\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 64 的第 1 片回包请求响应时延: 11.447702, 传输完成时延: 77.39417070802301\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 65 的第 1 片回包请求响应时延: 14.361961, 传输完成时延: 91.35327554214746\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 66 的第 1 片回包请求响应时延: 4.572569, 传输完成时延: 55.482984374975786\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 67 的第 1 片回包请求响应时延: 6.638175, 传输完成时延: 76.5968310830649\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 68 的第 1 片回包请求响应时延: 19.936595, 传输完成时延: 59.81371804093942\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 69 的第 1 片回包请求响应时延: 5.033213, 传输完成时延: 68.97196416696534\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 70 的第 1 片回包请求响应时延: 7.174712, 传输完成时延: 81.14681887510233\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 71 的第 1 片回包请求响应时延: 12.824959, 传输完成时延: 84.78778825001791\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 72 的第 1 片回包请求响应时延: 11.09626, 传输完成时延: 89.08478745794855\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 73 的第 1 片回包请求响应时延: 6.425819, 传输完成时延: 68.35492762480862\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 74 的第 1 片回包请求响应时延: 7.70108, 传输完成时延: 88.6965792500414\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 75 的第 1 片回包请求响应时延: 0.384964, 传输完成时延: 84.38052424998023\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 76 的第 1 片回包请求响应时延: 10.001273, 传输完成时延: 61.91754674981348\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 77 的第 1 片回包请求响应时延: 9.606839, 传输完成时延: 50.48885108297691\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 78 的第 1 片回包请求响应时延: 12.238151, 传输完成时延: 79.18801299994811\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 79 的第 1 片回包请求响应时延: 12.093623, 传输完成时延: 70.0196209999267\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 80 的第 1 片回包请求响应时延: 11.718375, 传输完成时延: 74.6520281250123\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 81 的第 1 片回包请求响应时延: 8.58229, 传输完成时延: 83.556586666964\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 82 的第 1 片回包请求响应时延: 7.689929, 传输完成时延: 64.61536904214881\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 83 的第 1 片回包请求响应时延: 17.983828, 传输完成时延: 86.93857550015673\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 84 的第 1 片回包请求响应时延: 6.121841, 传输完成时延: 91.10962570901029\n",
      "[INFO] [12-28 13:04:23] dataset.py:1727 [t:8120441664]: 数据 85 的第 1 片回包请求响应时延: 6.162251, 传输完成时延: 88.16282537509687\n"
     ]
    }
   ],
   "source": [
    "# 用户可以设置 service_model 为自己想要的模型名，来直接对数据进行批量推理，以 EB 4 为例\n",
    "result = ds.test_using_llm(service_model=\"ERNIE-Bot-4\")\n",
    "\n",
    "# 用户还可以设置 service_endpoint 来使用预置或自己的服务。\n",
    "result = ds.test_using_llm(service_endpoint=\"completions_pro\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果用户有异步请求的需求，还可以使用 `atest_using_llm` 来进行异步批量推理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: nest_asyncio in /Users/pengyiyang/miniconda3/envs/bce-qianfan-sdk/lib/python3.10/site-packages (1.5.8)\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] openapi_requestor.py:165 [t:8120441664]: async requesting llm api endpoint: /chat/eb-instant\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:230 [t:8120441664]: trying to refresh access_token for ak `BWMBe7***`\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:244 [t:8120441664]: sucessfully refresh access_token for ak `BWMBe7***`\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:33] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:34] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[INFO] [12-26 14:34:34] oauth.py:301 [t:8120441664]: no appid provided, using the latest one which id is 44352275\n",
      "[ERROR] [12-26 14:34:34] openapi_requestor.py:110 [t:8120441664]: api request failed with error code: 18, err msg: Open api qps request limit reached, please check https://cloud.baidu.com/doc/WENXINWORKSHOP/s/tlmyncueh\n",
      "[WARNING] [12-26 14:34:34] base.py:424 [t:8120441664]: got error code 18 from server, retrying... count: 0\n",
      "[ERROR] [12-26 14:34:34] openapi_requestor.py:110 [t:8120441664]: api request failed with error code: 18, err msg: Open api qps request limit reached, please check https://cloud.baidu.com/doc/WENXINWORKSHOP/s/tlmyncueh\n",
      "[WARNING] [12-26 14:34:34] base.py:424 [t:8120441664]: got error code 18 from server, retrying... count: 0\n",
      "[ERROR] [12-26 14:34:34] openapi_requestor.py:110 [t:8120441664]: api request failed with error code: 18, err msg: Open api qps request limit reached, please check https://cloud.baidu.com/doc/WENXINWORKSHOP/s/tlmyncueh\n",
      "[WARNING] [12-26 14:34:34] base.py:424 [t:8120441664]: got error code 18 from server, retrying... count: 0\n",
      "[ERROR] [12-26 14:34:34] openapi_requestor.py:110 [t:8120441664]: api request failed with error code: 18, err msg: Open api qps request limit reached, please check https://cloud.baidu.com/doc/WENXINWORKSHOP/s/tlmyncueh\n",
      "[WARNING] [12-26 14:34:34] base.py:424 [t:8120441664]: got error code 18 from server, retrying... count: 0\n",
      "[ERROR] [12-26 14:34:46] openapi_requestor.py:110 [t:8120441664]: api request failed with error code: 18, err msg: Open api qps request limit reached, please check https://cloud.baidu.com/doc/WENXINWORKSHOP/s/tlmyncueh\n",
      "[WARNING] [12-26 14:34:46] base.py:424 [t:8120441664]: got error code 18 from server, retrying... count: 0\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 0 的第 1 片回包请求响应时延: 9.91669949144125e-05, 传输完成时延: 0.4361692911479622\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 1 的第 1 片回包请求响应时延: 0.00023029185831546783, 传输完成时延: 13.562330374959856\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 2 的第 1 片回包请求响应时延: 0.0001822090707719326, 传输完成时延: 3.2577857091091573\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 3 的第 1 片回包请求响应时延: 6.087496876716614e-05, 传输完成时延: 3.044143709121272\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 4 的第 1 片回包请求响应时延: 0.000212125014513731, 传输完成时延: 6.462688042083755\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 5 的第 1 片回包请求响应时延: 9.037507697939873e-05, 传输完成时延: 17.96569495811127\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 6 的第 1 片回包请求响应时延: 9.608292020857334e-05, 传输完成时延: 1.5345318329054862\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 7 的第 1 片回包请求响应时延: 0.0001805000938475132, 传输完成时延: 11.2599307091441\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 8 的第 1 片回包请求响应时延: 0.000124374870210886, 传输完成时延: 9.617378416936845\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 9 的第 1 片回包请求响应时延: 0.00027716602198779583, 传输完成时延: 17.02750320895575\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 10 的第 1 片回包请求响应时延: 0.00019141682423651218, 传输完成时延: 1.4084208749700338\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 11 的第 1 片回包请求响应时延: 0.00021766708232462406, 传输完成时延: 13.772446250077337\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 12 的第 1 片回包请求响应时延: 0.00010100007057189941, 传输完成时延: 4.0809381660074\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 13 的第 1 片回包请求响应时延: 0.00014937506057322025, 传输完成时延: 13.38427250017412\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 14 的第 1 片回包请求响应时延: 9.195809252560139e-05, 传输完成时延: 15.08111112494953\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 15 的第 1 片回包请求响应时延: 9.766686707735062e-05, 传输完成时延: 6.732011415995657\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 16 的第 1 片回包请求响应时延: 4.337495192885399e-05, 传输完成时延: 6.044348082970828\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 17 的第 1 片回包请求响应时延: 0.0002174580004066229, 传输完成时延: 14.257962542120367\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 18 的第 1 片回包请求响应时延: 0.0005506670568138361, 传输完成时延: 5.014207749860361\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 19 的第 1 片回包请求响应时延: 0.00021900003775954247, 传输完成时延: 16.790142458863556\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 20 的第 1 片回包请求响应时延: 0.00013729184865951538, 传输完成时延: 11.54624329204671\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 21 的第 1 片回包请求响应时延: 0.0002012921031564474, 传输完成时延: 2.7028473331592977\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 22 的第 1 片回包请求响应时延: 0.00024916697293519974, 传输完成时延: 17.779967457987368\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 23 的第 1 片回包请求响应时延: 0.00015258393250405788, 传输完成时延: 1.6021012079436332\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 24 的第 1 片回包请求响应时延: 0.0002265418879687786, 传输完成时延: 3.7688526250422\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 25 的第 1 片回包请求响应时延: 0.0001652499195188284, 传输完成时延: 17.888612792128697\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 26 的第 1 片回包请求响应时延: 0.0002323330845683813, 传输完成时延: 2.4081481248140335\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 27 的第 1 片回包请求响应时延: 0.00016958406195044518, 传输完成时延: 18.44272620906122\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 28 的第 1 片回包请求响应时延: 0.00014483300037682056, 传输完成时延: 15.42698054201901\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 29 的第 1 片回包请求响应时延: 7.612491026520729e-05, 传输完成时延: 1.231050250120461\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 30 的第 1 片回包请求响应时延: 6.183399818837643e-05, 传输完成时延: 9.625307583017275\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 31 的第 1 片回包请求响应时延: 0.00011616712436079979, 传输完成时延: 4.094993165927008\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 32 的第 1 片回包请求响应时延: 0.00018983311019837856, 传输完成时延: 5.233883083099499\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 33 的第 1 片回包请求响应时延: 0.00015724985860288143, 传输完成时延: 7.552518415963277\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 34 的第 1 片回包请求响应时延: 0.00015262491069734097, 传输完成时延: 6.161027333931997\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 35 的第 1 片回包请求响应时延: 0.00010516704060137272, 传输完成时延: 9.624755625147372\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 36 的第 1 片回包请求响应时延: 0.00015070918016135693, 传输完成时延: 14.057984250131994\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 37 的第 1 片回包请求响应时延: 0.00030754110775887966, 传输完成时延: 7.006263750139624\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 38 的第 1 片回包请求响应时延: 0.0002348748967051506, 传输完成时延: 4.695738459005952\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 39 的第 1 片回包请求响应时延: 0.00015104212798178196, 传输完成时延: 12.560483541805297\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 40 的第 1 片回包请求响应时延: 5.941605195403099e-05, 传输完成时延: 15.110301333013922\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 41 的第 1 片回包请求响应时延: 0.0002359999343752861, 传输完成时延: 16.668715792009607\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 42 的第 1 片回包请求响应时延: 0.00011183391325175762, 传输完成时延: 7.621081541990861\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 43 的第 1 片回包请求响应时延: 0.00010037492029368877, 传输完成时延: 10.876424416899681\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 44 的第 1 片回包请求响应时延: 9.945896454155445e-05, 传输完成时延: 9.244093832792714\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 45 的第 1 片回包请求响应时延: 0.0005737920291721821, 传输完成时延: 12.409634459065273\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 46 的第 1 片回包请求响应时延: 0.00010912492871284485, 传输完成时延: 6.27830308303237\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 47 的第 1 片回包请求响应时延: 0.0001657919492572546, 传输完成时延: 5.84002795885317\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 48 的第 1 片回包请求响应时延: 9.612506255507469e-05, 传输完成时延: 8.757093333173543\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 49 的第 1 片回包请求响应时延: 0.00016129203140735626, 传输完成时延: 18.377021124819294\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 50 的第 1 片回包请求响应时延: 0.0002345831599086523, 传输完成时延: 13.18480695784092\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 51 的第 1 片回包请求响应时延: 0.0001538340002298355, 传输完成时延: 7.514169542118907\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 52 的第 1 片回包请求响应时延: 0.0001559578813612461, 传输完成时延: 15.129992125090212\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 53 的第 1 片回包请求响应时延: 0.0001790828537195921, 传输完成时延: 16.82620899984613\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 54 的第 1 片回包请求响应时延: 8.050003089010715e-05, 传输完成时延: 8.717674875166267\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 55 的第 1 片回包请求响应时延: 0.00021658302284777164, 传输完成时延: 14.968403583858162\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 56 的第 1 片回包请求响应时延: 0.00018400000408291817, 传输完成时延: 12.881917499937117\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 57 的第 1 片回包请求响应时延: 0.00014804210513830185, 传输完成时延: 7.2754291670862585\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 58 的第 1 片回包请求响应时延: 0.00025637494400143623, 传输完成时延: 0.8645364588592201\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 59 的第 1 片回包请求响应时延: 0.00010895892046391964, 传输完成时延: 8.992438541958109\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 60 的第 1 片回包请求响应时延: 0.0002785839606076479, 传输完成时延: 12.780613125069067\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 61 的第 1 片回包请求响应时延: 0.0002348329871892929, 传输完成时延: 12.55254629207775\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 62 的第 1 片回包请求响应时延: 0.00027183303609490395, 传输完成时延: 6.987949332920834\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 63 的第 1 片回包请求响应时延: 0.0002946250606328249, 传输完成时延: 9.742287250002846\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 64 的第 1 片回包请求响应时延: 0.0001487499102950096, 传输完成时延: 20.973076042020693\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 65 的第 1 片回包请求响应时延: 0.0002338751219213009, 传输完成时延: 5.578192416811362\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 66 的第 1 片回包请求响应时延: 0.0001752499956637621, 传输完成时延: 17.036143124802038\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 67 的第 1 片回包请求响应时延: 7.333303801715374e-05, 传输完成时延: 3.088283999823034\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 68 的第 1 片回包请求响应时延: 0.00021425005979835987, 传输完成时延: 11.216906207846478\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 69 的第 1 片回包请求响应时延: 0.000171333784237504, 传输完成时延: 14.13872450008057\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 70 的第 1 片回包请求响应时延: 0.0001682080328464508, 传输完成时延: 16.02978099999018\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 71 的第 1 片回包请求响应时延: 0.00022933399304747581, 传输完成时延: 18.18830620800145\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 72 的第 1 片回包请求响应时延: 0.0001812500413507223, 传输完成时延: 11.544364209054038\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 73 的第 1 片回包请求响应时延: 0.00014925003051757812, 传输完成时延: 7.3352779999841005\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 74 的第 1 片回包请求响应时延: 0.00022395816631615162, 传输完成时延: 12.315069500124082\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 75 的第 1 片回包请求响应时延: 0.0002524580340832472, 传输完成时延: 11.943211582954973\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 76 的第 1 片回包请求响应时延: 0.00015754206106066704, 传输完成时延: 20.286526583135128\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 77 的第 1 片回包请求响应时延: 0.0028433750849217176, 传输完成时延: 8.151763750007376\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 78 的第 1 片回包请求响应时延: 6.24998938292265e-05, 传输完成时延: 20.127325542038307\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 79 的第 1 片回包请求响应时延: 0.00017341598868370056, 传输完成时延: 5.660947999916971\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 80 的第 1 片回包请求响应时延: 0.00018366705626249313, 传输完成时延: 13.18093533301726\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 81 的第 1 片回包请求响应时延: 0.0003766249865293503, 传输完成时延: 7.076771792024374\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 82 的第 1 片回包请求响应时延: 0.00020150002092123032, 传输完成时延: 10.124220749828964\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 83 的第 1 片回包请求响应时延: 0.00017608399502933025, 传输完成时延: 11.421311415964738\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 84 的第 1 片回包请求响应时延: 0.00012091686949133873, 传输完成时延: 15.55511554190889\n",
      "[INFO] [12-26 14:34:54] dataset.py:1701 [t:8120441664]: 数据 85 的第 1 片回包请求响应时延: 0.00026520900428295135, 传输完成时延: 13.580984957981855\n"
     ]
    }
   ],
   "source": [
    "%pip install nest_asyncio\n",
    "import asyncio\n",
    "import nest_asyncio\n",
    "\n",
    "nest_asyncio.apply()\n",
    "\n",
    "result = asyncio.run(ds.atest_using_llm(service_endpoint=\"completions_pro\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "拿到的 `result` 对象也是一个 `Dataset` 对象，可以继续使用千帆 Python SDK 进行后续处理，或者直接保存到本地。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [12-28 13:05:47] dataset.py:981 [t:8120441664]: list local dataset data by 0\n",
      "[INFO] [12-28 13:05:47] dataset.py:525 [t:8120441664]: no destination data source was provided, construct\n",
      "[INFO] [12-28 13:05:47] dataset.py:337 [t:8120441664]: construct a file data source from path: output_file.csv, with args: {'is_download_to_local': False}\n",
      "[INFO] [12-28 13:05:47] data_source.py:280 [t:8120441664]: use format type FormatType.Csv\n",
      "[INFO] [12-28 13:05:47] dataset.py:233 [t:8120441664]: export as format: FormatType.Csv\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'prompt': '地球的自转周期是多久？', 'input_prompt': '地球的自转周期是多久？', 'llm_output': '地球的自转周期是**24小时**，即一天一夜。这是地球绕自身轴线旋转一周所需的时间。', 'expected_output': '大约24小时'}\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(result.list(0))\n",
    "      \n",
    "dataset_save_file_path = \"output_file.csv\"\n",
    "\n",
    "result.save(data_file=dataset_save_file_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 对模型进行批量推理\n",
    "\n",
    "除了对 `Service` 进行批量推理，我们也可以对 `Model` 进行批量推理\n",
    "\n",
    "在对 `Model` 进行批量推理时，请先确认用到的数据集已经在千帆平台上发布"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [01-05 16:31:19] dataset.py:466 [t:8120441664]: no data source was provided, construct\n",
      "[INFO] [01-05 16:31:19] dataset.py:350 [t:8120441664]: construct a qianfan data source from existed id: 44478, with args: {'is_download_to_local': False}\n",
      "[INFO] [01-05 16:31:20] dataset.py:150 [t:8120441664]: a cloud dataset has been created\n",
      "[INFO] [01-05 16:31:20] dataset_utils.py:353 [t:8120441664]: start to create evaluation task in model\n",
      "[INFO] [01-05 16:31:24] dataset_utils.py:315 [t:8120441664]: start to polling status of evaluation task 2817\n",
      "[INFO] [01-05 16:31:24] dataset_utils.py:322 [t:8120441664]: current eval_state: Pending\n",
      "[INFO] [01-05 16:31:54] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:32:25] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:32:55] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:33:25] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:33:56] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:34:26] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:34:57] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:35:28] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:35:58] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:36:28] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:36:59] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:37:29] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:37:59] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:38:30] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:39:00] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:39:30] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:40:01] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:40:31] dataset_utils.py:322 [t:8120441664]: current eval_state: Doing\n",
      "[INFO] [01-05 16:41:01] dataset_utils.py:322 [t:8120441664]: current eval_state: DoingWithManualBegin\n",
      "[INFO] [01-05 16:41:01] dataset_utils.py:340 [t:8120441664]: get result dataset id 44629\n",
      "[INFO] [01-05 16:41:01] dataset.py:466 [t:8120441664]: no data source was provided, construct\n",
      "[INFO] [01-05 16:41:01] dataset.py:350 [t:8120441664]: construct a qianfan data source from existed id: 44629, with args: {}\n",
      "[INFO] [01-05 16:41:02] data_source.py:1096 [t:8120441664]: start to fetch dataset cache because is_download_to_local is set\n",
      "[INFO] [01-05 16:41:02] data_source.py:697 [t:8120441664]: no cache was found, download cache\n",
      "[INFO] [01-05 16:41:03] data_source.py:589 [t:8120441664]: get dataset info succeeded for dataset id 44629\n",
      "[INFO] [01-05 16:41:03] data_source.py:596 [t:8120441664]: start to export dataset\n",
      "[INFO] [01-05 16:41:03] data_source.py:600 [t:8120441664]: create dataset export task successfully\n",
      "[INFO] [01-05 16:41:05] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:06] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:08] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:08] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:10] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:11] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:13] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:13] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:15] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:15] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:17] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:18] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:20] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:20] data_source.py:610 [t:8120441664]: export succeed\n",
      "[INFO] [01-05 16:41:20] data_source.py:565 [t:8120441664]: get export records succeeded for dataset id 44629\n",
      "[INFO] [01-05 16:41:20] data_source.py:581 [t:8120441664]: latest dataset with time2024-01-05 16:41:19 for dataset 44629\n",
      "[INFO] [01-05 16:41:20] data_source.py:626 [t:8120441664]: start to download dataset zip from url https://bj.bcebos.com/easydata-upload/export_local/%E8%AF%84%E4%BC%B0%E4%BB%BB%E5%8A%A1_model_run_LzzKiuZ4Q0_%E7%BB%93%E6%9E%9C%E9%9B%86_f23d10V1_20240105_164103.zip?authorization=bce-auth-v1%2F50c8bb753dcb4e1d8646bb1ffefd3503%2F2024-01-05T08%3A41%3A20Z%2F3600%2Fhost%2Fcb1ed680558c418f42272cbe1269b3a980db021e9140c4225eeb0d5c58eb34aa\n",
      "[INFO] [01-05 16:41:21] data_source.py:645 [t:8120441664]: download dataset zip to .qianfan_dataset_cache/36737/44629/1/bin.zip succeeded\n",
      "[INFO] [01-05 16:41:21] data_source.py:672 [t:8120441664]: unzip dataset to path .qianfan_dataset_cache/36737/44629/1/content successfully\n",
      "[INFO] [01-05 16:41:21] data_source.py:676 [t:8120441664]: write dataset info to path .qianfan_dataset_cache/36737/44629/1/info.json successfully\n",
      "[INFO] [01-05 16:41:21] data_source.py:725 [t:8120441664]: dataset cache is outdated, update cache\n",
      "[INFO] [01-05 16:41:22] data_source.py:589 [t:8120441664]: get dataset info succeeded for dataset id 44629\n",
      "[INFO] [01-05 16:41:22] data_source.py:565 [t:8120441664]: get export records succeeded for dataset id 44629\n",
      "[INFO] [01-05 16:41:22] data_source.py:581 [t:8120441664]: latest dataset with time2024-01-05 16:41:19 for dataset 44629\n",
      "[INFO] [01-05 16:41:22] data_source.py:596 [t:8120441664]: start to export dataset\n",
      "[INFO] [01-05 16:41:23] data_source.py:600 [t:8120441664]: create dataset export task successfully\n",
      "[INFO] [01-05 16:41:25] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:25] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:27] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:27] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:29] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:30] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:32] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:32] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:34] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:35] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:37] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:37] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:39] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:40] data_source.py:613 [t:8120441664]: export status: 1, keep polling\n",
      "[INFO] [01-05 16:41:42] data_source.py:605 [t:8120441664]: polling export task status\n",
      "[INFO] [01-05 16:41:42] data_source.py:610 [t:8120441664]: export succeed\n",
      "[INFO] [01-05 16:41:42] data_source.py:565 [t:8120441664]: get export records succeeded for dataset id 44629\n",
      "[INFO] [01-05 16:41:42] data_source.py:581 [t:8120441664]: latest dataset with time2024-01-05 16:41:40 for dataset 44629\n",
      "[INFO] [01-05 16:41:42] data_source.py:626 [t:8120441664]: start to download dataset zip from url https://bj.bcebos.com/easydata-upload/export_local/%E8%AF%84%E4%BC%B0%E4%BB%BB%E5%8A%A1_model_run_LzzKiuZ4Q0_%E7%BB%93%E6%9E%9C%E9%9B%86_f23d10V1_20240105_164122.zip?authorization=bce-auth-v1%2F50c8bb753dcb4e1d8646bb1ffefd3503%2F2024-01-05T08%3A41%3A42Z%2F3600%2Fhost%2Fe94091781afdfbb5dd8acbbd52697410f579db8959f78d5711eb0571823e9487\n",
      "[INFO] [01-05 16:41:43] data_source.py:645 [t:8120441664]: download dataset zip to .qianfan_dataset_cache/36737/44629/1/bin.zip succeeded\n",
      "[INFO] [01-05 16:41:43] data_source.py:672 [t:8120441664]: unzip dataset to path .qianfan_dataset_cache/36737/44629/1/content successfully\n",
      "[INFO] [01-05 16:41:43] data_source.py:676 [t:8120441664]: write dataset info to path .qianfan_dataset_cache/36737/44629/1/info.json successfully\n",
      "[INFO] [01-05 16:41:43] dataset.py:905 [t:8120441664]: list local dataset data by 0\n",
      "[INFO] [01-05 16:41:43] dataset.py:905 [t:8120441664]: list local dataset data by None\n",
      "[INFO] [01-05 16:41:43] dataset.py:905 [t:8120441664]: list local dataset data by 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'prompt': '人类的基本单位是什么？', 'input_prompt': '人类的基本单位是什么？', 'llm_output': '人类社会学界、群落、组织、团队、组织、机构、公司、政府、公司、家庭、家庭、军队、学校和家庭、村落、门派、门派、门派、门派、门派是人类的集合。\\n\\n\\n\\n\\n人类的基本结构层次单位是集合名词，人类的基本单位是集合名词，人类的基本单位是集合名词，人类的基本单位是集合名词，人类的基本单位是个人。\\n\\n\\n人类的基本单位是基本单位是个人。', 'expected_output': '人类'}\n"
     ]
    }
   ],
   "source": [
    "cloud_dataset_id = \"dataset_id\"\n",
    "\n",
    "qianfan_ds = Dataset.load(qianfan_dataset_id=cloud_dataset_id, is_download_to_local=False)\n",
    "\n",
    "result = qianfan_ds.test_using_llm(model_version_id=\"amv-qb8ijukaish3\")\n",
    "print(result[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 进阶能力\n",
    "\n",
    "在调用 `test_using_llm` 时，用户还可以传入一些额外参数来支持额外的功能，比如设置 Prompt 模板，设置人设字段，或者传入大模型调用时的超参数\n",
    "\n",
    "当对服务进行非对话类推理时，用户可以传入 `prompt_template` 参数来传递一个 Prompt 模板。`prompt_template` 是一个千帆 Python SDK 的 `Prompt` 对象，用户可以通过设置 `Prompt` 对象的 `template` 成员来自定义被用于推理的模板，模板渲染出来的内容将会被作为最终输入提交给大模型。以示例数据集为例，我们可以这么指定一个模板："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from qianfan.common import Prompt\n",
    "\n",
    "prompt = Prompt(template=\"请你就以下问题进行回答: {prompt}\")\n",
    "\n",
    "# 传递给函数\n",
    "result = ds.test_using_llm(service_model=\"ERNIE-Bot-4\", prompt_template=prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除此之外，用户还可以传入 `system_prompt` 参数来指定对话中大模型需要遵守的人设"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = ds.test_using_llm(service_model=\"ERNIE-Bot-4\", system_prompt=\"人设 prompt\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用户在进行批量推理时，还可以直接在 test_using_llm 中传入模型支持的超参数，例如我们可以这么设置模型的 `temperature` :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "result = ds.test_using_llm(service_model=\"ERNIE-Bot-4\", system_prompt=\"人设 prompt\", temperature=0.1)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "bce-qianfan-sdk",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
